API Catalog Maintenance Patterns

How APIs.guru Processes Its Growing Database of API Specifications

A TidyTuesday analysis examining catalog maintenance patterns on APIs.guru, revealing that 40% of APIs receive same-day processing when added to the database. The visualization explores how different providers’ APIs are managed in the catalog system and highlights the efficiency of APIs.guru’s internal processing workflows.

Figure 1: Two-panel chart analyzing APIs.guru catalog patterns from 2016-2023. Top: scatter plot with log scale y-axis showing days between addition and update, 1,011 red dots at 0.5 days (same-day processing), 1,518 gray dots scattered above (later updates). Bottom: horizontal bars ranking 15 providers by same-day processing percentage.

Steps to Create this Graphic

1. Load Packages & Setup

```{r}
#| label: load
#| warning: false
#| message: false
#| results: "hide"

## 1. LOAD PACKAGES & SETUP ----
suppressPackageStartupMessages({
if (!require("pacman")) install.packages("pacman")
pacman::p_load(
  tidyverse,  # Easily Install and Load the 'Tidyverse'
  ggtext,     # Improved Text Rendering Support for 'ggplot2'
  showtext,   # Using Fonts More Easily in R Graphs
  janitor,    # Simple Tools for Examining and Cleaning Dirty Data
  scales,     # Scale Functions for Visualization
  glue,       # Interpreted String Literals
  patchwork   # The Composer of Plots
  )})

### |- figure size ----
camcorder::gg_record(
  dir    = here::here("temp_plots"),
  device = "png",
  width  =  8,
  height =  10,
  units  = "in",
  dpi    = 320
)

# Source utility functions
suppressMessages(source(here::here("R/utils/fonts.R")))
source(here::here("R/utils/social_icons.R"))
source(here::here("R/utils/image_utils.R"))
source(here::here("R/themes/base_theme.R"))
```

2. Read in the Data

```{r}
#| label: read
#| include: true
#| eval: true
#| warning: false

tt <- tidytuesdayR::tt_load(2025, week = 24)

info_raw <- tt$api_info |> clean_names()
apis_raw <- tt$apisguru_apis |> clean_names()

tidytuesdayR::readme(tt)
rm(tt)
```

3. Examine the Data

```{r}
#| label: examine
#| include: true
#| eval: true
#| results: 'hide'
#| warning: false

glimpse(info_raw)
glimpse(apis_raw)
```

4. Tidy Data

```{r}
#| label: tidy
#| warning: false

### |-  tidy data ----
apis_minimal <- apis_raw |>
  left_join(info_raw, by = "name") |>
  select(name, added, updated, provider_name) |>
  mutate(
    added_date = as_date(added),
    updated_date = as_date(updated)
  ) |>
  filter(!is.na(added_date), !is.na(updated_date))

catalog_analysis <- apis_minimal |>
  mutate(
    days_between_add_update = as.numeric(updated_date - added_date),
    catalog_update_type = case_when(
      days_between_add_update == 0 ~ "Same-day catalog processing",
      days_between_add_update <= 7 ~ "Updated within week",
      days_between_add_update <= 30 ~ "Updated within month",
      days_between_add_update <= 365 ~ "Updated within year",
      days_between_add_update > 365 ~ "Long gap before catalog update",
      TRUE ~ "Other"
    ),
    same_day_processing = days_between_add_update == 0,
    days_for_viz = ifelse(days_between_add_update == 0, 0.5, days_between_add_update)
  ) |>
  filter(days_between_add_update >= 0)

# Calculate key statistics
total_apis <- nrow(catalog_analysis)
same_day_count <- sum(catalog_analysis$same_day_processing)
same_day_pct <- round(same_day_count / total_apis * 100, 1)
later_updated_count <- total_apis - same_day_count
later_updated_pct <- round(later_updated_count / total_apis * 100, 1)

# P2. Provider Analysis ----
provider_catalog_analysis <- apis_minimal |>
  mutate(
    days_between_add_update = as.numeric(updated_date - added_date),
    same_day_processing = days_between_add_update == 0
  ) |>
  filter(days_between_add_update >= 0) |>
  group_by(provider_name) |>
  filter(n() >= 3) |>
  summarise(
    total_apis = n(),
    same_day_apis = sum(same_day_processing),
    same_day_rate = round((same_day_apis / total_apis) * 100, 1),
    avg_days_to_update = round(mean(days_between_add_update), 1),
    .groups = "drop"
  ) |>
  filter(same_day_rate > 0) |>
  arrange(desc(same_day_rate)) |>
  head(15) |> # Top 15 by same-day processing rate
  mutate(
    provider_clean = str_trunc(str_to_title(provider_name), 35),
    provider_clean = fct_reorder(provider_clean, same_day_rate)
  )
```

5. Visualization Parameters

```{r}
#| label: params
#| include: true
#| warning: false

### |-  plot aesthetics ----
colors <- get_theme_colors(
  palette = c("#95A5A6", "#E74C3C", "#7F8C8D")
)

### |-  titles and caption ----
caption_text <- create_social_caption(
  tt_year = 2025,
  tt_week = 24,
  source_text =  "APIs.guru"
)

### |-  fonts ----
setup_fonts()
fonts <- get_font_families()

### |-  plot theme ----

# Start with base theme
base_theme <- create_base_theme(colors)

# Add weekly-specific theme elements
weekly_theme <- extend_weekly_theme(
  base_theme,
  theme(
    # Axis elements
    axis.text = element_text(color = colors$text, size = rel(0.7)),
    axis.title.x = element_text(color = colors$text, face = "bold", size = rel(0.8), margin = margin(t = 15)),
    axis.title.y = element_text(color = colors$text, face = "bold", size = rel(0.8), margin = margin(r = 10)),

    # Grid elements
    panel.grid.major.y = element_line(color = "gray50", linewidth = 0.05),
    panel.grid.minor = element_blank(),
    panel.grid.major.x = element_blank(),

    # Legend elements
    legend.position = "plot",
    legend.title = element_text(family = fonts$tsubtitle, color = colors$text, size = rel(0.8), face = "bold"),
    legend.text = element_text(family = fonts$tsubtitle, color = colors$text, size = rel(0.7)),

    # Plot margins
    plot.margin = margin(t = 15, r = 15, b = 15, l = 15),
  )
)

# Set theme
theme_set(weekly_theme)
```

6. Plot

```{r}
#| label: plot
#| warning: false

# P1. The API Graveyard Plot ----
p1 <- catalog_analysis |>
  ggplot(aes(x = added_date, y = days_for_viz)) +
  # Geoms
  geom_point(
    data = catalog_analysis |> filter(!same_day_processing),
    color = colors$palette[1], size = 1.5, alpha = 0.6
  ) +
  geom_point(
    data = catalog_analysis |> filter(same_day_processing),
    color = colors$palette[2], size = 2.5, alpha = 0.9
  ) +
  # Scales
  scale_y_log10(
    labels = comma_format(),
    limits = c(0.1, NA),
    breaks = c(0.5, 1, 10, 100, 1000)
  ) +
  # Annotations
  annotate("text",
    x = as.Date("2016-06-01"), y = 12,
    label = paste0("Gray dots: Updated later in catalog\n(", later_updated_count, " APIs • ", later_updated_pct, "%)"),
    color = colors$palette[1], size = 3.5, fontface = "bold",
    hjust = 0, vjust = 0
  ) +
  annotate("text",
    x = as.Date("2016-06-01"), y = 0.9,
    label = paste0("Red dots: Same-day catalog processing\n(", same_day_count, " APIs • ", same_day_pct, "%)"),
    color = colors$palette[2], size = 3.5, fontface = "bold",
    hjust = 0, vjust = 0
  ) +
  # Labs
  labs(
    title = "**API Catalog Maintenance Patterns**",
    subtitle = paste0(
      "**", same_day_pct, "%** of APIs have <span style='color:#E74C3C'>**same-day catalog processing**</span> when added to APIs.guru<br><br>",
      "<span style='color:#555555; font-size:10px'>**Days Between Addition and Catalog Update (log scale)**</span>"
    ),
    x = "Date Added to APIs.guru",
    y = NULL,
  ) +
  # Theme
  theme(
    plot.title = element_markdown(
      size = rel(1.6),
      family = fonts$title,
      face = "bold",
      color = colors$title,
      lineheight = 1.1,
      margin = margin(t = 0, b = 5)
    ),
    plot.subtitle = element_markdown(
      size = rel(0.95),
      family = fonts$subtitle,
      color = alpha(colors$subtitle, 0.9),
      lineheight = 1.2,
      margin = margin(t = 5, b = 10)
    ),
    panel.grid.minor = element_blank(),
    legend.position = "none"
  )

# P2. Provider Responsibility Breakdown Plot ----
p2 <- provider_catalog_analysis |>
  ggplot(aes(x = provider_clean, y = same_day_rate)) +
  # Geoms
  geom_col(fill = colors$palette[2], alpha = 0.7, width = 0.7) +
  geom_text(aes(label = paste0(same_day_rate, "% • (", same_day_apis, "/", total_apis, ")")),
    hjust = -0.1, size = 3, color = "gray50", fontface = "bold"
  ) +
  # Scales
  scale_y_continuous(
    limits = c(0, 124),
    expand = c(0, 0)
  ) +
  coord_flip() +
  # Labs
  labs(
    title = "**Catalog Processing Patterns by Provider**",
    subtitle = "Providers with highest rates of same-day catalog processing on APIs.guru",
    x = NULL,
    y = "% of APIs with Same-Day Catalog Processing",
  ) +
  # Theme
  theme(
    plot.title = element_markdown(
      size = rel(1.6),
      family = fonts$title,
      face = "bold",
      color = colors$title,
      lineheight = 1.1,
      margin = margin(t = 5, b = 5)
    ),
    plot.subtitle = element_markdown(
      size = rel(0.95),
      family = fonts$subtitle,
      color = alpha(colors$subtitle, 0.9),
      lineheight = 1.2,
      margin = margin(t = 5, b = 10)
    ),
    panel.grid.major.y = element_blank(),
    panel.grid.minor = element_blank(),
    axis.text.y = element_text(size = 9),
    plot.caption = element_text(size = 8, color = colors$palette[3])
  )

# Final plot -----
combined_plot <- p1 / p2 +
  plot_layout(heights = c(1.2, 1))

combined_plot <- combined_plot +
  plot_annotation(
    caption = caption_text,
    theme = theme(
      plot.caption = element_markdown(
        size = rel(0.6),
        family = fonts$caption,
        color = colors$caption,
        hjust = 0.5,
        margin = margin(t = 10)
      )
    )
  )
```

7. Save

```{r}
#| label: save
#| warning: false

### |-  plot image ----  
save_plot_patchwork(
  plot = combined_plot, 
  type = "tidytuesday", 
  year = 2025, 
  week = 24, 
  width  =  8,
  height =  10
)
```

8. Session Info

Expand for Session Info

R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 22631)

Matrix products: default


locale:
[1] LC_COLLATE=English_United States.utf8 
[2] LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
 [1] here_1.0.1      patchwork_1.3.0 glue_1.8.0      scales_1.3.0   
 [5] janitor_2.2.0   showtext_0.9-7  showtextdb_3.0  sysfonts_0.8.9 
 [9] ggtext_0.1.2    lubridate_1.9.3 forcats_1.0.0   stringr_1.5.1  
[13] dplyr_1.1.4     purrr_1.0.2     readr_2.1.5     tidyr_1.3.1    
[17] tibble_3.2.1    ggplot2_3.5.1   tidyverse_2.0.0 pacman_0.5.1   

loaded via a namespace (and not attached):
 [1] gtable_0.3.6       httr2_1.0.6        xfun_0.49          htmlwidgets_1.6.4 
 [5] gh_1.4.1           tzdb_0.5.0         yulab.utils_0.1.8  vctrs_0.6.5       
 [9] tools_4.4.0        generics_0.1.3     parallel_4.4.0     curl_6.0.0        
[13] gifski_1.32.0-1    fansi_1.0.6        pkgconfig_2.0.3    ggplotify_0.1.2   
[17] lifecycle_1.0.4    compiler_4.4.0     farver_2.1.2       munsell_0.5.1     
[21] codetools_0.2-20   snakecase_0.11.1   htmltools_0.5.8.1  yaml_2.3.10       
[25] crayon_1.5.3       pillar_1.9.0       camcorder_0.1.0    magick_2.8.5      
[29] commonmark_1.9.2   tidyselect_1.2.1   digest_0.6.37      stringi_1.8.4     
[33] labeling_0.4.3     rsvg_2.6.1         rprojroot_2.0.4    fastmap_1.2.0     
[37] grid_4.4.0         colorspace_2.1-1   cli_3.6.4          magrittr_2.0.3    
[41] utf8_1.2.4         withr_3.0.2        rappdirs_0.3.3     bit64_4.5.2       
[45] timechange_0.3.0   rmarkdown_2.29     tidytuesdayR_1.1.2 gitcreds_0.1.2    
[49] bit_4.5.0          hms_1.1.3          evaluate_1.0.1     knitr_1.49        
[53] markdown_1.13      gridGraphics_0.5-1 rlang_1.1.6        gridtext_0.1.5    
[57] Rcpp_1.0.13-1      xml2_1.3.6         renv_1.0.3         vroom_1.6.5       
[61] svglite_2.1.3      rstudioapi_0.17.1  jsonlite_1.8.9     R6_2.5.1          
[65] fs_1.6.5           systemfonts_1.1.0 

9. GitHub Repository

Expand for GitHub Repo

The complete code for this analysis is available in tt_2025_24.qmd.

For the full repository, click here.

10. References

Expand for References

1. Data Sources:

• TidyTuesday 2025 Week 24: [API Specs](https://github.com/rfordatascience/tidytuesday/blob/main/data/2025/2025-06-17